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ABSTRACT , . . 

In a framiswork of cognitive theory, an exploraitory 
study examined eye movements of learners studying print materials 
combinijng text and illustrations. Eye movement patterns were used as 
indicators of the qognitive strategies employed. Both design and 
learner variables were emphasized. Twenty-four graduate education 
students were randomly assigned to each of two page layout 
treatments. Design variables were text complexity (simple or cortplex) 
and page layout (either picture first and then words or words before 
picture). Learner variables were sex, cognitive style (field 
dependence/independence), and prior knowledge. The dependent variable 
was visual attention indicated by eye movement data including 
duration, transitions, and sequences. Subjects studied the inaterial 
while fitted with head-mounted pye movement detecting equi^>ineht. Of 
the design variables, the complexity factor was most dete^rminative of 
learne-r strategy. Of the learner variables, cognitive style and sex 
were reliably associated , with learner strategies, but prior knowledge 
had no effect on strategy. In an analysis of the complex page, 
results revealed a tendency for females to favor th^ given- order and 
males the reverse; Six references are listed.^ (LMM) > 
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Visu a l 'Attention to 'Picture and" Word Materials as Ir;)fluenced t?y 
Characteristics of the Learners and Design.-of the Materials. 



Purpose, The^long ranrje Qo^l of thi's^ program of research is to 
ifnprove the match . between the design of instructional materials 
and the characteristics of learners. Particular attention'is on 
the cognitijv/e strategies which skilled 1 earners empl oy in the , 
studyy/3f materials eombini ng words and pictures- - , ^ 

Theoretical interest centers on the fact that recent 
research on verbal -and imaginal processing suggests th'at word^ 
and pictures are cognitively disparatej, requi<^rin'g different kinds 
and amounts of proce-ssing. This implies potential cognitive 
-problems where words and pictures are mixed indiscriminately in 
instructional materials. • ; " - 

\ What follows was an exploratory study of^ how skilled 
learner^ cope^with materials combining words and pictures;, e-g-,' 
textbooks. 

Baci; ground. . This study was done in a framewor^< of cognitive 
theory. It specifically examined the eye movements of learners 
studying print materials combining text and illustrations. The 
eye-movement patterns ^were seen as useful i^ndi cators of the . 
cognitive strategies employed. (There is ample precedent for 
this interpretation of eye-movement datan^e.g.^ Snow^ 1968; Just 
and Carpenter 5 l^SO; Farnham-Di ggery and Greggj, 1975) Strategies 
were; inferred ffom the ^^.ff^erential duration of^ attention to word 



and picture areas and from the numbers and^atterns of 
transi ti bns" between areas- * 

The writer ascribes to a constructi vi st model of re^dingj 
where- reading compreherwei on involves the reader s search for 
ineu- ling from the- author^ s woi7ds as well, as his diagrams, graphs, 
pictures, etc- Meaning i s constructed by the reader both from 

what the material provides and what the^reader provides from 

■% ' - ' ■ . ■ " • 

hi 5/ her own wor 1 d knowl edge- The ski 1 1 ed <uni ver si ty-1 evel ) 



reader'^s prior knowledge includes strategi e'.s f or studying and 
schema for interpreting and rememberi ng vari ouc kinds of 
materials, e-g-, stori es, • exposi tory text, research arti cl es, - 
sc^ience diagrams, pictorial scenes, etc. 

Research to vdate on reading comprehension has-^xami/ned 
primarily the words (prose leai^ning)s to a lesser extent the 
illustrations (picture learning), and rarely both in interaction 
(textbook learning)- This both explains the preponerance of 
prose learning concepts in what follows and justifies the curl^ent 
study of strategies involving both text and illustration- 

* 

Much has bf?en made qf the di fferences . between, word 
processing (Linear, digital, symbolic, left brain, visual and 
auditory modalities, learned skill) and picture processing 
(si mul taneous"", analogic, concrete, right brain, visual modality, 
less learning)- . Clearly there ere processes speci f i c to each, . . 
but- on a- typical page combining bot^J they must somehow be 
integrated- A macrostruc^ure (theme, gist) must be constructed 
Jj^hat^includ^s both- Ap^^gntly, these di verse -elemen'ts ar^ 



integrated primarily at a higher semantic levv, . * 

^ Reading theories^ differ in degree of emphasis on what 

information is given in the text as compared \ v<hat is brought 

. ■ . . ♦ 

^ to it by the reader Those emphasizing /heaning-i -'^-the-text are 

remindful of the position of many audi ovi sual protessi onal • 

i.e. J the meaning is in thp medium^ especially the picHiures- 

Both emphasise the given stimul u*j, whether word-.or picture. In 

• • 

contrast, the readi^ng construct i vi st theories which 'emphasize the 

„ interaction of reader dhar acter-* st'i cs^, conteVit, and\Qrint are 

remindfjul of the aptitude <or trait), treatment interaction 

theorfsts in i nstructional development. Both emphasize the 

differential outcomes from what's given depending on the 

1 earner " s pri or knowl edge> i nterest s, skills, etc' ^ ^. ^ 

• j ' ' ■ , ' ' 

The above suggests the two sets of pertinent variables which 

•this study investigated, ile., what s -gi ven <design variables> ' 

. • ■• • 

and what the learner brings. to it i< learner vari abl es) . One of 

\ . V ^ . . ',• ■ . 

the design variables (what^s given) often citeB in readlngU 

comprehension research is conte>:tj, i.e., the verbal cpnt'fex t on V^v^^^^ 

the page. In the 'present analysis of realworld instructional 

materials it is apparent that words inr textbooks often provide 

context for pictures, »e.g., i nterp)"eti ve captions,^ and .pictures' 

- often provide context for words, e.g., pictured examples of 

concepts. Often the intended relation between text and 

illustration appears ambiguous. Where oneo appears above the 

** 

ottier on a page, do reader ^ assume that the one provides a 
■(Context for what follows? The question of page-layout orderj, 

V 166 



i-e.'t which comes fi'rst^ words or pictures, was e>:amined in this 

study Di fferent cogniti ve str ategi es 'were expected fdr the two 

page-layouts, UIP and PW, but the direction of those differences^ 

>jf ♦ ■ 
was riot- predicted, y > 

Another of the design variables common in reading research 

is complexity, which is often oper ati onal i zed' according ^to a 

readability formufa involvir^g word length or frequency^, sentence 

length, etc. ^ Complexity has also been variously operati onal ized 

in picture studies by number of . f igures, ' degree of* realism, etc. 
' ft 

♦ •■ • ' 

In this study complexity was opei^ati bnal i zed irKtwo ways: grade 

level of t)ie materials and length of ' sentences.^ It was predicted 

that cognitive strategics for complex materials would include 

longer study durations ^nd more transitions between areas than 

for simple materials.^ ^ ' • 

.Comm(^ measures of learner characteri sti cs^i n reading 
studiesVare prof i ci encyv^t^^ts, e.g^, vocabulary or 

comprehension. The present study using graduate^udents assumed 
high reading and study skills. However, the sample was divided ' 

sex because it was hypothesi zed that word/pi cture study 
strategies of males and females might differ. The evidence *that 
women tend to have higher verbal skills (Lipsanb Col will, 1978) 
as well as higher grades in general (Maccoby and Jackl in, 1974) 
was the basi s f or ' pr edi cti ons of areater attenti on to words and 
longer overall study duration for women thari men. . ^ 

Another learner ^character i sti c studied here was cognitive 



styl^, Specifically Wel£^(^pehdence-,f ield independence. The 



eyij^dence that field dependents tend to. be morp global and field 
independents mor'e analytici.A <W'iti<in!, et al , 1974i led to the » 
prediction.^that ^jiobal field dependents ^^ul d fol 1 ow <be 
dependent on"^ .the §iven pattern of. information on the page^ 
whereas the more analytical field i ndep-endents would tend '^o 
deviate from (be' icndependent of) the given pattern". 

World kno^;iedge is considered to be an important factor in 
reading comprehension.^ 1x5 the present study* an attempt was made 
to estimate the prior knowledge of the 1 earner s 'about the subject 
matter^ biological sfience. It was predicted that the mor^ 
knowledgeable learners wopl d -have study strategi es which were 
shorter in duration and which involved fewer transi ti ons between 
areas than would less knowledgeable learners. 



Procedure. A convenience sample of 24 students' was selected from 

— ' ■ • • 

a graduate class in Education such that half were males and half 
f emale&. "^Half of each 5e>: grouping* were randomly assi gned tdr each 
of two; page-1 ayout treatments. ^ - - ^ ' 

The design variables were complexity at two levels^ simple 

v.s. compl'exj. and page layout at twolevel^s^ picture first and 

,j - ■* . ' 

then words <PW) v.s. words first then^picture <WP) . ^See Fig. 1 

for an example of a PW layout. , The simple material wa^ -Prom an 

8th grad^ text.5 while 1>he complex was from a scientific , research . 

'journal. They contained comparable numbers bf^^;WordSj but 

• ' #"1 

sentences -in the complex mfe»teri al averaged about twice^as long. 
The learner variables were sex, cognitive style (field 
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\ir, Z The ranfe of hearing in firUS trit Uu. The frequency line h dnvn biiFiriilaHMany. fnm» unu Herts 
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■ ' * • ' .' 

KccoGMt/SMG I'MtnATORK itv Ear: simnd pulses are ctmiptised of 40 kHr 
Tmk riTi:ii Is TJHt SiviTCM y^ioiies. ihe crickers abditmen and legs 

In ii» wtirld a rntkci heir* wn tinly t»ih^ abruptly veer ii» ihe righr. in free-flight this 
CI crkkets. it hears |J«iicnrial priilatiiri. uMutd propel the crit ket away fnim the 
Cr ickeis are sensitive tii a surprisnigly^ wide siiund source. l*hus, 5 kHz-Mtnes elicit pos- 
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is onlv a narrow one considered in light of The sign of the niiiveiiients n»a^es bcliav- 
the iiisect's auditory capabilitieii Figure 2 inral sense: 6 kH.z n the carrier of fre- 
shows a freciuencv range from zero Hi to t{nenc)' of the calling simg tif Tft^giyllta 
100.000 Hz—from Jnfrasound to ultra- orMin*rtu; female crickets are attracted to 
sqund. Cr^ickets arc-Sensitive over a good this fret4uency>Fony kHz is in the ultra- 
pan of this range; compare this tvith the sonic range and occurs in the vncalizatbns 
auditor)' sensitivity of humans; tvhich spans ' prttduced by insectivorous bats (priffini 
a range of 50 Hz to 15,000 Hz. 1»74); crickets attempting to escape from 

Hgure-SA-C shows diagrams taken from echolocatihg bats would be expected to 
phtrtographs of living crickets rcsptmdlng react to40kHz aversively. ; 
to acoustie stiniuli! Inihe absence of sound Th^ frequenq- sensitivity of steering be- 
a cricket flies whh a svninieirical flight |H)S- hi vior can be ascertained by cNamining a 
ture. with it^lungitudinal btidy axis per- I' :havioral tuning cur\e (Rj*. 30) made by 
fecily siraiglit. When a series ^f sound nitasuringt the threshold «uiiid intensmes 
ptilses consistifig of pure 5 kHz loncs I nnuired tei eJkit a>h»nutactic response as 
played from a spwlier <m the cricket's lef ; a function of the itme frciiucncy. T. oemn- 
the in«ea b^nds iu abcltimen and legs »o io« is most mponsive toitmes in «ne range, 
thi lefs. a rodder-iike anion thai would 4-6 kHzi with peaW sensitivity ai_5 kHz: a 
prtipci ihe insect tw%^rd the speaker, wercis^ond area of sensitivity occurs m the ul- 
k J»»e insen not tethered; However, when the , tcasouiid. from 2a-100 kHz. 

EIUC BEST COPY AVAIUBLE^^^^^.^^^^^ F^^ ■ . ; 



dependent/field i ndependent )j ^ and^^rior knowledge. . - 

/ I • • * 

The^^dependent varXN^ble was visual, attention as indi(;^ated by 
eye-movement data, specifically the amount of tiine spent 
attending to each area c^f' the materials (duration) , the^nLVDber of 

shifts between areas (transi tri,dns) ji and the pattern of • 

V - - / • ■ ^ ' ■ •• 

t^ransitions (sequences). " r . 

Subjects were instructed to.st}-\dy the material' as though it'' 

had beeo assigned ^or- tHem- to understand an=d^ remember . They were^ 

then fit;ted with head-amounted' eye-mbyemerit-defbctin^ equipment 

(NAC Eyemark IV). They wer^ allowed as much td me as needed to 

study the mapfcerials. - ; . ' . . ^ ' 

■« ' . - . I . ; ■ V • . 

Subjects then completed a brief questionnaire concerning^ the 

• . • . ■ ^ ' V 

rel c.t i ve. ami 1 i ari ty and difficulty of the materials and ^he 
number of related course^ (bi ol ogy) whicH they had. taken. Thpy 
then took a cognitive style test, the GEFT (Grotrp Embedded 
Figures Test). 

Raw data r ecord^. consi sted of vi deotape recordings showing , 
the stimulus'^ a subject was studying plus a superimposed marken 

r 

• . . ■ . . ■ N ' . . 

'which indicated just where 'on the stimulus the^ubject was , 

looking at any moment. The four, pages of' stimuli were, divided 

/ • • . • < , ' ^ r . ^. 

into 30 ^gnificant ar eas, -i . e. , either figures in the pictures 

or c;aptions and par-agra'phs in.the te>:t. Read out. from the ^ 

videotape 'records', i^er'e ithe duratijbn :of attention 'to each area and 

the number and sequencelbiF transitions between areas- Twenty of 

the 24 reef or ds were usaale- , ' • 



Results: Overal 1 . A 3-way ANOVA of ttee ef-i^etts of SeM and .Layout 
. . ^ V 

.{independent measures) and Complexity (repeated measDre) on total 

■ -1" 

transitions revealed the predi cted .si gnf i cant main effect for 
complexity^ p<.00l5 but no efffect for sex or layout. SObjects T~ J 
made more transi ti ors * i n studying the complex material than the\ 



si/npl e. 




Analysis of the effects- on tot^l duration revealed . \ 
significant di.f f erences^- as expected^ foir both sex>.p<.0045 and 
complexity 3 p<.001.^ Females attended longer overall to the study 
materials than did^ales. "All subjects studied the compl ex 
mater i al s^ more than the simple-* / » 

I ,. '• ■ • .•, ■■■■ ■■■ 

There was no main effect for layout <PW v.s. WP) nor for, 
prior knowledge. Analyses of the data relative to familiarity of 
the content as wel 1 as nunfber of biology courses taken revealed 
.no significant rel ati onshi ps to cogni tive strategy . 

'3 ■ • V' ■■ ■■ :,. ■ . 

r There -was ^ si gni fi cant positive correlatipn between 

cognitive style (GEFT) and nufiiber of transiti9ns. for complex 

materials only (Spearman" p'<. 034. * Kendal 1 p<.037). So, a /rtedian 

split was made of the subjects by GEFT score and two groups 

formed^ higher scor^ers (Field Independents) and lower scorers 

(Fi el Dependents) . An ANOVA reyealed a significant two-way 

interaction between cogriitiVe style and xromplexi ty^ p< .003, for 

transitions. W.hile subjec^^s of both cogni ti ve^^tyTes made more 

transitions on complex materials thc^n on simple, the field 

independents made a larger adjustment tp complexity, i.e. they 

*- ■ . ^ . ■ ^ ^ , » ,L • 

ma(Je"rel.ativ.ely fewer t^^^itions for the simple and relatively 



more tce^nsitions for thje complex materials thran did the field 
dependents.- See Fig. Z. : . ' 

R esults; Compl e>! Ha ter i al ; Because sub jects^♦strategi es seemed 
most influenced by- the cbmpl e>i i ty factor , it . was -deci ded to i 
analyze the most complex . page further i.' A* 3-^way ANOVA^ Page' 
Layourt by Se>: by Sequence^ revealed' a si gnif leant . main effect for* 

sequence^ p<.05. There V^eres of courBe^ more ; traasi ti ons in the 

»■ -J ■ ■ . . 

given sequence <top down) ±han in the reversed. Howeye;=^5 there 

■ - ■ ■ ' . 

was. a si gni f i cant 2-way interaction between se>; and sequence, 

p<.af. the females favoring the given order and the males the 

reversed. :. Thi s was -further modif ietl by a 3-way^lnter^1ii on 

involving pag^ layout, wftich revejal ed. that the tendency for -males 

to make more reverse order- transitions' was limited to the. WP 

layput condition (words first, then pi ctures) - .See Fig. 3.^ 

The above interactions suggest a very interesting 
possibility^- Assume that .a .compatible match of cognitive, 
strategy and page layout would show up as a largely straight - 
through (top down) attenti on . sequence j i.e.., a minimum nuniber of 
transitions and most of these in the given order. It follows, 
according tq-^this assumption, that the. most compat i bl e 1 ayput fb|"' 
males was 9^ and for females WP. Bee Fig.- 3 for the curves with • 



a low overall number of transitions, .and with more- in the given 
than 'reversed order. 



A further analysis was made of attenti on patterns to the 
five pictorial areas, two related verbal areas, and one caption 
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area on the complex page- A 2~way ANOVA^ Se>: by Layout ^ revealed 
no differences in attention to picture areas but a significant^ 
difference in attention to verbal areas^ p*/-05- As predicted ^ 
women made more transitions to verbal areas and maintained 
attentipn there longer than males. For the caption area there 
was a significant effect for layoutj, there bei hg v more transi ti oas 

to the caption., p<-01p and longer durations on th'^ caption^ 

'i ' 

p<.055 for the PW layout than for the WP layout. |This is 
understandable because the PW layout put the caption in the . 
middle of the page where it was more noticeable^ JLhereas the WP 
layout put it at the bottom- ^ 



Another analysis of the complex page^ Cogni ti ve Style by 
Layout, revealed no di f f erences f or the verbal areas but 
significant differences for pictorial areas^^<l05 for -durations 
and p<.01 for transitions. Field i ndependents 1 ooked longer and 
more often at picture areas on the complex page than did field 

/ - ■ ■ N ■ 

dependents. This was' not predicted. 

' ■ •■ 

I ^ 

Conclusions . Of the design variables;, the compl ex i ty factor was 

ddearly the most determinative of learnllr strategy. Of the 

\- ■ > • 

learner variables, prior knowledge had no effect on strategy. 

■ 

Perhaps the measures of prior knowledge were inadequate. The 

} * ■ • 

other learner variables, cognitive style and sex, were reliably. , 

associated with learner strategies. 

These main effects and interactions with cogni ti ve . styl ^ and 
sex were largely as predicted from prior research and theory. 

- \ • 175 . ■ \ ■ ^ 



However, there were some surpri^es^, e.g.? the effect of cognitive 

style on strategy adaptati orf^ to instr'ucti onal material 

complexity, and the e-f f e^ of se>; on strategies for sequencing 

attention to words and- pictures. : ' ^ . 

/ ^ • / _ ■ . ■■ 

CI early 5 more such studies are* needed to confirm or 

di sconfi rm these conclusions.' However? the results do suggest 
that the design of science materials for graduate students may 
'need to take account of twoplearner variables, se>: and cognitive 
^ ' style, and two design variables, complexity and picture-word 

1 ayout • 

/ 
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